The present invention relates to a motion picture editing method and a motion picture editing apparatus editing motion picture data played and output from at least one motion picture file stored on a recording medium and having a motion picture data body arranged in temporal order along with time management information.
Multimedia information employed for the so-called streaming playback playing data while receiving the same is described with reference to FIG. 23. Each multimedia information, independently stored as a single completed file per certain unit, is formed by header control information indicating the attribute, the property etc. of the overall multimedia information included in this file and a multimedia data body.
The header control information includes control information related to the overall multimedia information stored in the file, such as a file identifier (file ID) for specifying the file, the length of the multimedia data body, a time required for playing the overall file, the average/maximum bit rate, the types and the number of media included in the multimedia data body, the dependence relation between the media (an upper layer and a lower layer of hierarchized information, a right channel, a left channel, a center and a surround of an audio etc.), the association between the media (Japanese/English/French/voice guidance and subtitles of images and voices, Japanese/English of close captions etc.) and the like.
The header control information also includes information related to systematic control, related to only partial media included in the multimedia information but necessary for playing the file, such as the frame size/resolution of images, a coding method (identification of ITU-Recommendation H.261, H.262, H.263, H26L, ISO standard 11172 (MPEG-1), 13818-2 (MPEG-2), 14496-2 (MPEG-4) or the like, for example), parameters (the profile, the level, selection of an option mode etc.) in the coding system and the like.
Further, the header control information also includes information for storing the position of the multimedia information, such as index information specifying random-accessible positions, proper positions of temporally continuous multimedia information, stored marker information and the like.
The index information is utilized for deciding an access position halfway through the multimedia information, deciding playback object information for playing information in a skipping manner at a high speed or the like. Specifically, an intra-frame/field coded frame (intra-coded frame/field) or the like is specified as the index information when coded motion picture data is referred to.
This is because the intra-frame/field coded frame is a proper access point since the same is independently coded with no supplementation from precedent and subsequent information (frames) or the like and can be individually decoded and played while subsequent motion picture information can be sequentially decoded and played on the basis thereof.
When intra-frame coded frames are periodically inserted as in the GOP (Group of Pictures) structure in the coding system such as MPEG-1 or MPEG-2, for example, the periodic intra-frame coded frames (heads of GOPs) can be specified with the index information respectively. Further, the quantity of information of the index information can be reduced by providing only indices specifying skipped intra-frame coded frames as one for two GOPs or one for three GOPs.
Thus, when the index information specifies only periodic intra-frame coded frames, high-speed playback can be readily implemented at a constant rate by merely tracing the index information in high-speed playback.
When scene change or large motion/change is coded by automatically inserting detected output or an intra-frame coded frame, the index information may also specify the automatically inserted intra-frame coded frame, in addition to the aforementioned periodic intra-frame coded frames. In this case, random-accessible positions can be set in more detail although periodicity of positions specified by the index information is not guaranteed.
The data body of the multimedia information is arranged and stored in order of playback times in such a state that various types of media such as motion pictures, voices/audios etc. are intermixed. The order of storage is decided only with reference to the times, regardless of the types of the media.
While the data may be stored along the order in filming/recording for implementing storage in the order of playback times, the order of storage is decided in consideration of processing (decoding) delays between reading and outputting information, which vary with the media. When simultaneously filmed/recorded motion picture and voice are to be simultaneously played, the motion picture having a larger decoding delay is stored in advance by this delay time.
The data body is stored in temporal order as described above, and hence the data may simply be sequentially read from the head of the file, decoded and thereafter output when the same is played. When a server keeps a file storing multimedia information and transmits the data of the file through the Internet in response to access from a user, for example, the burden on the server can be extremely effectively lightened due to the aforementioned characteristic since the server may simply sequentially read and send the data body.
As described later, a time stamp indicating a read/playback time is appended to each packet of the data body, in order to control the timing of reading and playback. The server is required to send the data body at the time according to the time stamp value of the read data body packet.
This access mode of the multimedia information is referred to as streaming transfer (streaming service).
As to the data body of the multimedia information, input information may be stored as such, or the information may be compressed and coded by the aforementioned coding system. Further, the data body is divided into packets of a proper size, and stored as paketed information. Therefore, the data body of the multimedia information in the file is formed by a packet train of the information of each medium.
The packets of this data body are created and stored for the information of the respective media, and a single packet basically includes only data of a single medium. The lengths of the packets may be fixed or variable. Further, the lengths (the maximum lengths or average lengths in the case of the variable lengths) of the packets may completely vary with the media, or may be substantially similar to each other regardless of the types of the media.
When the packets have fixed lengths, the lengths are decided in response to systematic conditions such as sector lengths, record lengths, track lengths etc. of recording media, for example. When the quantity of information (bit rate) per hour with respect to the data of the media is constant, the quantity of information coincides with time intervals and hence the packet lengths can be set in response to temporal conditions.
When the packets have variable lengths, the unit of packeting can be decided in response to the properties of the data (coded information) of the media. The; simplest unit is matched with the unit of coding/decoding/playback output processing such as one coded unit frame for voices, one frame/field for motion pictures or the like.
In the aforementioned example, however, the quantity of information is disadvantageously too small for voices or too large for motion pictures. Therefore, the data body is packeted by combining a plurality of coded unit frames with each other for voices or further dividing coded information of one frame/field for motion pictures.
The unit for dividing coded information of motion pictures may be matched with a delimiter in the coded data structure such as a GOB (Group Of Blocks) in the coding system of ITU-Recommendation H.261, a GOB or an adaptive slice of Recommendation H.263, a slice of Recommendation H.262|ISO Standard 13818-2 (MPEG-2), a slice (video packet) of Standard 14496-2 (MPEG-4) or the like.
Further, a packet header indicating the attribute of the packet is affixed to each packet. The packet header includes the packet length, a flag indicating whether or not the position is random-accessible, the time stamp for time management etc. The time stamp indicates the time to send the packet to a client, i.e., the time to start processing on the packet.
In the format of a multimedia information file X shown in FIG. 23, the header control information is arranged and stored at the head, followed by the multimedia data body. When the file is read from the head in playback, the header information is obtained first so that the whole situation of the data body can be grasped, a system necessary for playback can be set and initialized, and an operation mode can be set.
If the data body is stored at the head followed by the header control information to the contrary, the data body can be immediately accessed and read when system control and mode setting are unnecessary (no control parameter is necessary, the mode is previously known, etc.) in a simple playback system, for example.
Further, part of the header control information may be stored at the head, followed by the data body and the remaining part of the header control information. When index information, marker information and the like are not utilized, these data may be arranged at the back of the data body so that unnecessary information can be completely ignored in playback.
A playback apparatus for the aforementioned multimedia information is now described with reference to FIG. 24 and FIG. 25. FIG. 24 is a functional block diagram showing a conventional playback apparatus for multimedia information, and FIG. 25 is a flow chart showing playback operation in the conventional playback apparatus for multimedia information.
Referring to FIG. 24, the conventional playback apparatus includes a time stamp read part 1 reading a time stamp included in each packet header from a desired multimedia information file, a playback timer 2 starting counting a timer time on the basis of a playback instruction received from a user, a compare part 3 comparing the time stamp read in the time stamp read part 1 with the timer time (playback reference time) counted by the playback timer 2, and a playback control part 4 outputting the multimedia information file when the timer time is identical to or leads the time stamp as a result of comparison in the compare part 3.
The operation of this playback apparatus is now described with reference to the flow chart of FIG. 25. When the user instructs the playback apparatus to play the multimedia information file, the playback timer 2 starts counting the timer time. Further, the playback timer 2 reads the time stamp included in each packet header from the target multimedia information file (step 21), and acquires the current timer time (step 22).
Then, the compare part 3 compares the time stamp read at the step 21 with the timer time acquired at the step 22 (step 23), and when the timer time is identical to or larger than the time stamp, the playback control part 4 starts playing and outputting the multimedia information data (step 24). When the timer time is smaller than the time stamp, the process returns to the step 22 and repeats the aforementioned operation.
The playback apparatus is authorized to play and output the packet part to which the time stamp is appended only when the time stamp read in the time stamp read part 1 is identical to or smaller than the timer time counted in the playback timer 2, whereby the data can be output from the head of the file along the order of storage in response to the timer time.
However, the aforementioned conventional playback apparatus for multimedia information plays and outputs the multimedia data body on the basis of the result of comparison of the time stamp appended in order of playback times in the file and the timer time, and hence, when the multimedia data body is edited by temporal cut and paste such as extraction, division and connection, a new edited multimedia data body must be created and stored independently of the unedited multimedia data body. Therefore, extremely large areas are disadvantageously required as a working area for editing and a storage area for the result of editing.
In order to readily edit and change motion picture data without creating new motion picture data from a motion picture file, the following method is conceivable:
When only a part A and a part B are to be extracted, played and output from the multimedia information file X shown in FIG. 23, an instruction for playing the part B (a start time stamp 6 and an end time stamp 8) subsequently to the part A (a start time stamp 1 and an end time stamp 2) is supplied. When the motion picture data is played while reading the time stamps on the basis of this instruction, however, it follows that the part A is started to be played and displayed when the playback time of the playback timer reaches 1, no part is displayed while the timer time is 3 to 6 and the part B is started to be played and displayed when the timer time reaches 6 as shown in FIG. 26, and the part A and the part B cannot be continuously played immediately after the playback instruction.
The playback time for arbitrarily extracted partial data is decided by w the timer time (playback reference time) of the playback timer and cannot a be changed with respect to the timer time, and hence the data cannot be edited by temporal cut and paste employing extraction, division, connection and the like.
The present invention has been proposed in consideration of the aforementioned problems, and an object of the present invention is to provide a motion picture editing method and a motion picture editing apparatus capable of readily editing a multimedia data body arranged in temporal order along with time stamps by temporal cut and paste employing extraction, division, connection and the like.
The motion picture editing method according to the present invention edits motion picture data played and output from a motion picture file stored on a recording medium and having a motion picture data body formed by arranging time management information and motion picture data corresponding to the said time management information in temporal order.
The motion picture editing method previously specifies at least one playback range in the motion picture file and a playback time of the playback range, appends information for managing the specified playback range and the playback time of the playback range to an area other than the motion picture data body on the recording medium as edit information, and outputs motion picture data of the specified range in the motion picture file at the specified time on the basis of the time management information and the edit information.
For example, time management information (time stamp) included in the motion picture data body, a physical storage position in the motion picture file, a packet number, the serial number of index information, a frame/field number, an arbitrary playback time or the like can be employed for specifying the playback range.
Preferably, the motion picture editing method changes the time management information in the output motion picture data on the basis of the edit information, and creates a new motion picture file.
More preferably, the motion picture editing method appends edit information including information for managing the playback time for the output motion picture data to an area other than the motion picture data body on the recording medium, and creates a new motion picture file.
According to another aspect of the present invention, the motion picture editing method edits motion picture data played and output from a motion picture file stored on a recording medium and having a motion picture data body formed by arranging time management information and motion picture data corresponding to the said time management information in temporal order. The motion picture editing method previously specifies at least one playback range in the motion picture file and a playback time of the playback range, appends information for managing the specified playback range and the playback time of the playback range to an area other than the motion picture data body on the recording medium as first edit information, defines an edit result in the motion picture file on the basis of the time management information and the first edit information, further specifies at least one playback range in the edit result and a playback time of the playback range, appends information for managing the specified playback range and the playback time of the playback range to an area other than the motion picture data body on the recording medium as second edit information, and outputs motion picture data of the specified range in the motion picture file at the specified time on the basis of time management information in the edit result and the second edit information.
According to still another aspect of the present invention, the motion picture editing apparatus edits motion picture data played and output from a motion picture file stored on a recording medium and having a motion picture data body formed by arranging time management information and motion picture data corresponding to the time management information in temporal order. The motion picture editing apparatus includes means specifying a desired playback range and a playback time of the playback range, means extracting specific information of the specified playback range and playback time information, means creating edit information on the basis of the extracted specific information and playback time information and recording the edit information in an area other than the motion picture data body on the recording medium, and playback means playing the playback range at the specified time on the basis of the edit information.
According to a further aspect of the present invention, a motion picture playback apparatus plays a motion picture file stored on a recording medium and having a motion picture data body formed by arranging time management information and motion picture data corresponding to the time management information in temporal order. The motion picture playback apparatus includes edit information read means reading edit information appended to an area other than the motion picture data body on the recording medium and including information for managing a playback time of the motion picture data, time management information read means reading time management information in the motion picture data, playback time adjustment means carrying out a prescribed operation on the time management information read in the time management information read means and calculating a specified playback time on the basis of the information for managing the playback time read in the edit information read means, compare means comparing the specified playback time calculated in the playback time adjustment means with a time counted by a playback timer, and control means outputting motion picture data in the motion picture file at the specified time on the basis of a result of comparison in the compare means.
According to a further aspect of the present invention, the motion picture editing method appends edit information including information for managing a playback time of motion picture data to a motion picture data body stored on a recording medium and formed by arranging time management information and motion picture data corresponding to the time management information in temporal order, and creates a new motion picture file. The motion picture editing method appends the edit information to an area other than the motion picture data body on the recording medium.